Face and Gesture Based Human Computer Interaction

In this paper, we present a face and gesture based human computer interaction (HCI) system. We combine head pose and hand gesture to control the system. We can identify the positions of the eyes and mouth, and use the face center to estimate the pose of the head. Moreover, we introduce a technique for automatic gesture area segmentation and orientation normalization of the hand gesture. The user does not need to keep gestures in upright position and the system segments and normalizes the gestures automatically. The experimental results show that the proposed approach is accurate with gesture recognition rate of 93.6%. Also, the user can control multiple devices, including robots simultaneously through a wireless network.